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S ' Abstract 

cN ; 

We consider probabilistic automata on infinite words with acceptance defined by parity conditions. We consider 
three qualitative decision problems: (i) the positive decision problem asks whether there is a word that is accepted 
with positive probability; (ii) the almost decision problem asks whether there is a word that is accepted with 
probability 1; and (Hi) the limit decision problem asks whether for every e > there is a word that is accepted 
with probability at least 1 — e. We unify and generalize several decidability results for probabilistic automata 
over infinite words, and identify a robust (closed under union and intersection) subclass of probabilistic automata 

for which all the qualitative decision problems are decidable for parity conditions. We also show that if the 
CJ , input words are restricted to lasso shape (regular) words, then the positive and almost problems are decidable 

for all probabilistic automata with parity conditions. For most decidable problems we show an optimal PSPACE- 
complete complexity bound. 
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1 Introduction 

Probabilistic automata. The class of probabilistic automata for finite words was introduced in the seminal work 
of Rabin |16| as an extension of classical finite automata. Probabilistic automata on finite words have been 
extensively studied (see the book ITSl on probabilistic automata and the survey of [5]). Probabilistic automata 
on infinite words have been studied recently in the context of verification and analysis of reactive systems f^ 
milllTj. We consider probabilistic automata on infinite words with acceptance defined by safety, reachability, 
Biichi, coBiichi, and parity conditions, as they can express all commonly used specifications (like safety, liveness, 
fairness) of verification. 

Qualitative decision problems. We consider three qualitative decision problems for probabilistic automata on 
infinite words Ql [TTl : given a probabilistic automaton with an acceptance condition, (i) the positive decision 
problem asks whether there is a word that is accepted with positive probability (probability > 0); (ii) the almost 
decision problem asks whether there is a word that is accepted almost-surely (with probability 1); and (iii) the limit 
decision problem asks whether for every e > there is a word that is accepted with probability at least 1 — e. The 
qualitative decision problems for probabilistic automata are the generalization of the emptiness and universaUty 
problems for deterministic automata. 

Decidability and undecidability results. The decision problems for probabilistic automata on finite words have 
been extensively studied |[T5l l5l. and the main results establish the undecidability of the quantitative version of the 
decision problems (where the thresholds are a rational < A < 1, rather than and 1). The undecidability results 
for the qualitative decision problems for probabilistic automata on infinite words are quite recent. The results of [H 
show that the positive (resp. almost) decision problem is undecidable for probabilistic automata with Biichi (resp. 
coBiichi) acceptance condition, and as a corollary both the positive and almost decision problems are undecidable 
for parity acceptance conditions (as both Biichi and coBiichi conditions are special cases of parity conditions). The 
results of fT| also show that the positive (resp. almost) decision problem is decidable for probabilistic automata 
with coBiichi (resp. Biichi) acceptance condition, and these results have been extended to the more general case 
of stochastic games with imperfect information in ||3l and |[T2l . The positive and almost problems are decidable 
for safety and reachability conditions, and also for probabilistic automata over finite words. For all the decidable 
almost and positive problems for probabilistic automata PSPACE-complete bounds were established in ||7]|6l. It 
was shown in UJJ that the limit decision problem is undecidable even for probabilistic finite automata, and the 
proof can be easily adapted to show that the limit decision problem is undecidable for reachability, Biichi, coBiichi 
and parity conditions (see HI for details). 

Decidable subclasses. The root cause of the undecidability results is that for arbitrary probabilistic automata and 
arbitrary input words the resulting probabilistic process is complicated. As a consequence several researchers have 
focused on identifying subclasses of probabilistic automata where the qualitative decision problems are decidable. 
The work of lH presents a subclass of probabilistic automata, namely hierarchical probabilistic automata (HPA), 
and show that the positive and almost problems are decidable for Biichi and coBiichi conditions on HPAs. The 
work of [11] presents a subclass of probabilistic automata, namely #-acyclic automata, and show that the limit 
reachability problem is decidable for this class of automata over finite words. The two subclasses HPA and #- 
acyclic automata are incomparable in expressive power. 

Our contributions. In this work we unify and generalize several decidability results for probabilistic automata 
over infinite words, and identify a robust subclass of probabilistic automata for which all the qualitative decision 
problems are decidable for parity acceptance conditions. For the first time, we study the problem of restricting the 
structure of input words, as compared to the probabilistic automata, and show that if the input words are restricted 
to lasso shape words, then the positive and almost problems are decidable for all probabilistic automata with parity 
acceptance conditions. The details of our contributions are as follows. 

1. We first present a very general result that would be the basic foundation of the decidability results. We 



introduce a notion of simple probabilistic process: the non-homogeneous Markov chain induced on the 
state space of a probabilistic automaton by an infinite word is simple if the tail cr-field of the process has 
a particular structure. The structure of the tail a-field is derived from Blackwell-Freedman-Cohn-Sonin 
decomposition-separation theorem H |9l [iTl on finite non-homogeneous Markov chains which generalizes 
the classical results on homogeneous Markov chains. 

2. We then show that if we restrict the input words of a probabilistic automaton to those which induce simple 
processes, then the positive and almost decision problems are decidable for parity conditions. We establish 
that these problems are PSPACE-complete. 

3. We study for the first time the effect of restricting the structure of input words for probabilistic automata, 
rather than restricting the structure of probabilistic automata. We show that for all ultimately periodic 
(regular or lasso shape) words and for all probabilistic automata, the probabilistic process induced is a 
simple one. Hence as a corollary of our first result, we obtain that if we restrict to lasso shape words, then 
the positive and almost decision problems are decidable (PSPACE-complete) for all probabilistic automata 
with parity conditions. However, the limit decision problem for the reachability condition is still undecidable 
for lasso shape words, as well as for the Biichi and coBiichi conditions. 

4. We then introduce the class of simple probabilistic automata (for short simple automata): a probabilistic 
automaton is simple if every input infinite words induce simple processes on its state space. This seman- 
tic definition of simple automata uses the decomposition-separation theorem. We present a structural (or 
syntactic) subclass of the class of simple automata, called structurally simple automata, which relies on the 
structure of the support graph of the automata (the support graph is obtained via subset constructions of 
the automata). We show that the class of structurally simple automata generalizes both the models of HPA 
and ^-acyclic automata. Since HPA generalizes deterministic automata, it follows that structurally sim- 
ple automata with parity conditions strictly generalizes w-regular languages. We show that for structurally 
simple automata with parity conditions, the positive and almost problems are PSPACE-complete, and the 
limit problem can be decided in EXPSPACE. Thus our results both unify and generaUze two different re- 
sults for decidability of subclasses of probabilistic automata. Moreover, we show that structurally simple 
automata are robust, i.e., closed under union and intersection. Thus we are able to identify a robust subclass 
of probabilistic automata for which all the qualitative decision problems are decidable for parity conditions. 
From our structural characterization it also follows that given a probabilistic automaton, it can be decided in 
EXPSPACE whether the automaton is structurally simple. 

In this paper we use deep results from probability theory to establish general results about the decidability of 
problems on probabilistic automata. We present a sufficient structural condition to ensure semantic notions (of 
an induced probabilistic process being simple) coming from probability theory in the context of probabilistic 
automata. The proofs omitted due to lack of space are given in appendix. 

2 Preliminaries 

Distributions. Given a finite set Q, we denote by A(Q) the set of probability distributions on Q. Given a G A((5), 
we denote by Supp(a) the support of a, i.e. Supp(a) = {g € Q | ci{q) > 0}. 

Words and prefixes. Let E be a finite alphabet of letters. A word ti; is a finite or infinite sequence of letters from 
S, i.e., tt; € S* or ty E E'^. Given a word w = oi, a2... G TI^ and i G N, we define w{i) = ai, and we denote by 
u;[l..i] = oi, ..., flj the prefix of length i of w. Given j > i, we denote by w[i..j] = ai, ..., aj the subword of w 
from index i to j. An infinite word it; G S'^ is a lasso shape word if there exist two finite words pi and p2 in S* 
such that ui = pi • pg ■ 

Definition 1 (Finite Probabihstic Table (see (15))). A Finite Probabihstic Table (FPT) is a tuple T = 
(Q, S, {MajagS) a) where Q is a finite set of states, S is a finite alphabet, a is an initial distribution on Q, 



and the Ma, for a G S, are Markov matrices of size \Q\, i.e., for all q,q' a Q we have Ma{q, q') > and for all 
q eQwe have J2g'eQ ^''^a{q, q') = 1- 

Distribution generated by words. For a letter a G S, let 5{q, a){q') = Ma{q, q') denote the transition probability 
from q to q' given the input letter a. Given /3 G A((5), q ^ Q and p G S*, let 6{f3,p){q) be the probability, 
starting from a state sampled accordingly to /3 and reading the input word p, to go to state q. Formally, given 
p = ai, ..., a„ G S*, let Mp = Ma, ■ Ma, ■ ... ■ Ma„. Then ,5(/3, p){q) = Y^^^^q P{q') ■ Mp{q', q). We often write 
6{l3,p) instead of Supp(5(/9,p)), for simplicity: 6{/3,p) is the set of states reachable with positive probability 
when starting from distribution f3 and reading p. As well, given H C Q,we write 5{H, p) for the the set of states 
reachable with positive probability when starting from a state in H sampled uniformly at random, and reading p. 

Homogeneous and non-homogeneous Markov chains. A Markov chain is a sequence of random variables 
Xq,Xi,X2, ..., taking values in a (finite) set Q, with the Markov property: P(X„_|_i = x\Xi = xi,X2 = 
X2, . . . , Xn = Xn) = P(X„+i = x\Xn = x„). Givcn n G N, the matrix M„ of size \Q\ such that for all q,q' G Q 
we have Mn{q, q') = P(X„_|_i = q'\Xn = q) is the transition matrix at time n of the chain. The Markov chain is 
homogeneous if M„ does not depend on n. In the general case, we call the chain non-homogeneous. 

Induced Markov chains. Given a FPT with state space Q, given G Q Q and p = oq, ..., am-i G S* such that 
^{G, p) C G, we define the Markov chain {X„}„gN induced by (G, p) as follows: the initial distribution, i.e. the 
distribution of Xq, is uniform on G; given i G N, Xj+i is distributed according to 5{Xi, a^ ^od m)(— )■ Intuitively, 
{Xn\ne^ is the Markov chain induced on the FPT when reading the word p'^ . 

Probability space and ir-field. A word w G S'^ induces a probability space (17, J", P'"): Q. = Q^ is the set of 
runs, T is the u-field generated by cones of the type Cp = \r ^ Q^ \ r [1.. |p|] = p} where p G Q*, and P"' is the 
associated probability distribution on il. See [ 19] for the standard results on this topic. We write {X^}„£n for the 
non-homogeneous Markov chain induced on Q by w, and given n G N let p^ be the distribution of X^ on Q: 

Given g G Q, p'iliq) = P'"[{r G ^ \ r{n) = q}] 

The (7-field T is also the smallest ir-field on Q. with respect to which all the X^ , n G N, are measurable. For 
all n G N, let Tn = S{Xn,Xl^^-^, ...) be the smallest a-field on Q with respect to which all the Xf , i > n, 
are measurable. We define /"oo = PlnGN-^"' called the tail a-field of {X^}. Intuitively, an event F is in /"oo if 
changing a finite number of states of a run r does not affect the occurrence of the run r in F. 

Atomic events. Given a probability space (il, T, P) and F G /", we say that F is T -atomic if P(F) > 0, and for 
all F' G J" such that F' C F we have either P(F') = or P(F') = P(F). In this paper we will use atomic events 
in relation to the tail c-field of Markov chains. 

Acceptance conditions. Given a FPT, let F C Q be a set of accepting (or target) states. Given a run r, we denote 
by Inf (r) the set of states that appear infinitely often in r. We consider the following acceptance conditions. 

1. Safety condition. The safety condition Safe(F) defines the set of paths that only visit states in F; i.e., 
Safe(F) = { ((70, gi, • • •) I Vi > 0. g, G F }. 

2. Reachability condition. The reachability condition Reach(F) defines the set of paths that visit states in F at 
least once; i.e., Reach(F) = { (go, qi, ■ ■ ■) I 3i > 0. (/j G F }. 

3. Buchi condition. The Biichi condition Buchi(F) defines the set of paths that visit states in F infinitely often; 
i.e., Buchi (F) = { r | Inf(r) n F / }. 

4. coBUchi condition. The coBiichi condition coBuchi(F) defines the set of paths that visit states outside F 
finitely often; i.e., coBuchi(F) = { r [ Inf(r) C F }. 

5. Parity condition. The parity condition consists of a priority function p : Q — > N and defines the 
set of paths such that the minimum priority visited infinitely often is even, i.e.. Parity (p) = { t \ 
min(p(Inf (r))) is even }. Biichi and coBiichi conditions are special cases of parity conditions with two 
priorities (priority set { 0, 1 } for Biichi and { 1, 2 } for coBiichi). 



Probabilistic automata. A Probabilistic Automaton (PA) is a tuple A = (T, $) where T is a FPT and $ is an 
acceptance condition. 

Decision problems. Let ^ be a PA with acceptance condition <I> : il — ;■ {0,1}. We consider the following 
decision problems. 

1. Almost problem: Whether there exists ^« G S"^ such that P^(<1>) = 1? 

2. Positive problem: Whether there exists w e T,'^ such that P^(<1>) > 0? 

3. Limit problem: Whether for all e > 0, there exists w G S'^ such that P^(<1>) > 1 - e? 
Proposition [T] summarizes the known results from ifTllSl [771171161. 

Proposition 1. Given a PA with an acceptance condition $, the following assertions hold: 

1. The almost problem is decidable (PSPACE-complete) for ^ = safety, reachability, BUchi, and undecidable 
for $ = co-BUchi and parity. 

2. The positive problem is decidable (PSPACE-complete) for ^ = safety, reachability, co-BUchi, and undecid- 
able for $ = BUchi and parity. 

3. The limit problem is decidable (PSPACE-complete) for $ = safety, and undecidable for $ = reachability, 
BUchi, co-BUchi, and parity. 

3 Simple Processes 

In this section we first recall the decomposition-separation theorem, then use it to decompose the tail a-field of 
stochastic processes into atomic events. We then introduce the notion of simple processes, which are stochastic 
processes where the atomic events obtained using the decomposition-separation theorem are non-communicating. 

3.1 The Decomposition Separation Theorem and tail cr-flelds 

The structure of the tail a-field of a general non-homogeneous Markov chain has been deeply studied by math- 
ematicians. Blackwell and Freedman, in [4], presented a generalization of the classical decomposition theorem 
for homogeneous Markov chains, in the context of non-homogeneous Markov chains with finite state spaces. The 
work of Blackwell and Freedman has been deepened by Cohn f9l and Sonin {YT\, who gave a more complete 
picture. We present the results of ||4l|9l[l7l in the framework of jet decompositions presented in [17|. 

Jets and partition into jets. Kjet is a sequence J = { J(}igN» where each Ji Q Q. A tuple of jets (J^, J^, ..., J"^) 
is called a partition of Q^ into jets if for every n G N, we have that J^,j}i,...,J^ is a partition of Q. The 
Decomposition-Separation Theorem, in short DS-Theorem, proved by Cohn ||9l and Sonin ifTTl using results of 
101, is given in Theorem [T] We first define the notion of mixing property of jets. 

Mixing property of jets. Given a FPT A, a jet J = {JjjjgN is mixing for a word w if: given X^ ,n > the 
process induced on Q by w, given q, q' G Q, and a sequence of states {gijjeN such that for alH > we have 
qi G Ji, given m G N, if lim„P"'[X;f = q^\X^ = q]>0 and lim„P'"[X;f = q^ \ x:^ = q'] > 0, then we have: 



'P-[X- = (?„|X»G4AX-=g'] 

Intuitively, a jet is mixing if the probability distribution of a state of the process, conditioned to the fact that this 
state belongs to the jet, is ultimately independent of the initial state. This extends the notion of mixing process on 
homogeneous ergodic Markov chains, on which the distribution of a state of the process after a number of steps is 
close to the stationary distribution, irrespective of the initial state. 

Theorem 1 (The Decomposition-Separation (DS) Theorem jHlllini). Given a FPT A = {Q, H, {Ma}a&:,oi), 
for all w G T,^ there exists cG{1,2,...,|Q|} and a partition ( J^, J^, ..., J'^) ofQ^ into jets such that: 



1. With probability one, after a finite number of steps, a run r G enters into one of the jets J^, k G 
{ 1, 2, . . . , c } and stays there forever 

2. For all k G {1,2, ... ,c} the jet J^ is mixing. 

Theorem [Uholds even if S is infinite: it is valid for any non-homogeneous Markov chain on a finite state space. 
In this paper we will focus on finite alphabets only. 

Remark. We note that for alH G {1, 2, . . . , c}, either fJ-niJn) -^n^oo or there exists Aj > such that for n large 
enough fJ-n{Jn) > ^i- Indeed, if ^Jf (J^) -/^n^oo but there exists a subsequence of {fJ-n{Jn)}n&N which goes to 
zero, then a non zero probability of runs enter J^ and leave it afterward infinitely often, which contradicts the first 
point of Theorem [T] Thus, we can always assume that there exists A > such that for alH G { 1, 2, . . . , c }, forn 
large enough, we have jJ-^iJu) > ^- If this is not the case, we just merge the jets J* such that iJ,^{J^) —>n-^oo 
with J°, which does not invalidate the properties of the jet decomposition stated by Theorem [T] 

For the following of the section, we fix if G S'^ and a partition J^,J^,...,J^ of Q'^ as in the DS Theorem. 
Given z G { 1, 2, . . . , c } and n G N, let: 

< = {r G I r{i) G J^}, and r^ = UNen <^n>N < 

We now present a result directly from our formulation of the DS Theorem (the result can also be proved using 
more general results of |l9l). 

Proposition 2. For all i G { 1, 2, . . . , c }, the following assertions hold: (1) r^ G J^oo. i-S-> Ti° i^ <^ tail a-field 
event; (2) t^ is Too-atomic; i.e., t?° is an atomic tail event; and (3) P"'(IJi=i ''"oo) ~ -'- 

The fact that the r^ are atomic sets of Too means that all the runs which belong to the same r^ will satisfy the 
same tail properties. Intuitively, a tail does not depend on finite prefixes. Several important classes of properties 
are tail properties, as presented in fTOl : in particular any parity condition is a tail property. 

3.2 Simple processes characterization with jets 

Definition 2. Let {X^ }neN be a process induced on Q by a word w G S'^, and let fi^ be its probability distribution 
on Q at time n. We say that {fi^}n£N i^ simple if there exist A > and two sequences {An}n&n cind {Bn}n&n of 
subsets of Q such that: 

• Vn G N, An, Bn is a partition ofQ 

. Vn G N, Vg G A^, ^C{q) > A 

The second point of the following proposition shows that the tail c-field of a simple process can be decomposed 
as a set of "non-communicating" jets. Intuitively, a jet is non-communicating if there exists a bound A^ G N such 
that after time N, if a run belongs to the jet, it will stay in it for ever with probability one. The following proposition 
is a reformulation of the notion of simple process in the framework of jets decomposition. 

Proposition 3. Let w G T,^, and suppose that the process {^5^}„gN induced on Q is simple. Then there exists a 
decomposition ofQ^ into jets, J^, J^, ..., J'^, and N £ N, which satisfy the following properties: 

L For all n > N, alii G {1,2, ... ,c} and all q G J^, we have Unig) > A. 

2. For all i £ {1,2, ... ,c} and all n2 > rii > N we have 5{Jn^ , w^^+i) ^ J^j- 

J. fJ'ny'^n) ^n— i-oo U. 

4. Each jet ,P, i G {1,2, ... ,c} is mixing. 

4 Decidable Problems for Simple Processes and Lasso shape Words 

In this section we will present decidable algorithms (with optimal complexity) for the decision problems with 
the restriction of simple processes, and for lasso shape words. 



4.1 Decidable problems for simple processes 

We first define the simple decision problems that impose the simple process restriction. Given an acceptance 
condition $, we consider the following problems: 

1. Simple almost (resp. positive) problems: Does there exist tj; G S'^ such that {/xJ^jnGN is simple and 
p^($) = 1 (resp. P^($) > 0)? 

2. Simple limit problem: For all e > 0, is there tD € S'^ such that {jjL^]n&i is simple and P^(<1>) > 1 — e? 

Proposition |4] shows that the decidability and undecidability results of Proposition [T] concerning the positive, 
almost, and limit safety and reachability problems still hold when we consider their "simple process" version. 
Propositions [5] and |6] are more interesting as they show that the almost and positive parity problem become decid- 
able when restricted to simple processes. Finally, Proposition |7] shows that the "hmit" decision problems remain 
undecidable even when restricted to simple processes. 

Proposition 4. The simple almost (resp. positive) safety and reachability problems are PSPACE-complete, as well 
as the simple limit safety problem. The simple limit reachability problem is undecidable. 

Proposition 5. The simple almost parity problem is PSPACE-complete 

Proof. (Sketch). The proof relies on the following equivalent formulation. 

Equivalent formulation. In the following, p : Q — ^ N is a parity function on Q, and <1> = Parity (p). We prove 
that: (1) There exists w ^ TI^ such that the induced process is simple and P^(<1>) = 1 if and only if (2) There 
exists G Q Q and pi,p2 G S* such that G = 5{a, pi), d{G, P2) ^ G, and the runs on the Markov chain induced 
by (G, P2) satisfy $ with probability one. We show in the appendix that the properties can be verified in PSPACE 
and also present a PSPACE lower bound. 

We show the equivalence (2)44>(1). The way (2)^(1) is direct, since we will show in Section 14.21 that the 
process induced by a lasso shape word on any automaton is always simple. We prove that (1)^(2). Let w = 
oi, ..., aj, ... be such that the induced process is simple and P^($) = 1. Using Proposition [3l let J°, J^, ..., J™ be 
the decomposition of Q'^ into jets and let Nq £N, A > be such that: 
. Vn > iVo, Vi e { 1, 2, . . . , c }, V(7 G J^: Pn{q) > A. 

• Vi € { 1, 2, . . . , c }, for all ns > ni > A^o, we have 5(4^ , <J+i) C 4^. 

• Each jet J*, i G { 1, 2, . . . , c } is mixing. 

Without loss on generality, since Q is finite, taking A^o large enough, we can assume that the vector of sets of states 
{Jpf , ..., Jff ) appears infinitely often in the sequence {(J^, ..., J^)}„gN. As well, without loss on generality, we 
can assume that for all n > Nq and alH G { 1, 2, . . . , c }, all the states in J^ appear infinitely often among the sets 
J^, for m > Nq. Let i G { 1, 2, . . . , c }. Given q G Q, let 

<&g = {r G il I g G Inf(r) and p(q) = min p{q')} 

ij'elnf(r) 

Clearly, for all q G Q, ^q £ Too- Since Q is finite, there exists qi £ Q such that P(r^ n $g.) > 0. By Proposition 
|2j T^ is atomic, hence r^ C <I)^.. Since the runs of the process satisfy the parity condition with probability one, 
p{qi) must be even. Moreover, for all n > Nq and all q G J^, we must have p{q) > p{qi). Indeed, such a q 
appears an infinite number of times in the sequence J^, by hypothesis, and always with probability at least A. 

Since r^ C <I)q., there exists rrij G N such that for all q £ J- '\ there exists m < nii such that 6{q, w[Nq + 
1..7n]){qi) > 0. We define m = maxj^j i_2,...,c} "^i' and m' >m such that 



Taking pi=w[O..Nq] and p2 = w[Nq + l..A^o + ^^'] completes the proof. Indeed, when starting from the initial 
distribution, after reading pi, we arrive by construction in one of the sets Jj^ , with i E { 0, . . . , c }. Starting 
from this state q, if the word p2 is taken as input, we go to set J^ , ^, with probability one, visit qi with positive 
probability, and do not visit any state with probability smaller that p{qi). This implies that when starting from q 
and reading p2, we visit qi with probability one, hence the result. D 

Proposition 6. The simple positive parity problem is PSPACE-complete. 

A corollary of the proofs of Propositions |5] and [6] is that if the simple almost (resp. positive) parity problem is 
satisfied by a word, then it is in fact satisfied also by a lasso shape word. 

Proposition 7. The simple limit Biichi and coBUchi problems are undecidable. 

From the propositions of this section we obtain the following theorem. In the theorem below the PSPACE- 
completeness of the limit safety problem follows as for safety conditions the limit and almost problem coincides. 

Theorem 2. The simple almost and positive problems are PSPACE-complete for parity conditions. The simple 
limit problem is PSPACE-complete for safety conditions, and the simple limit problem is undecidable for reacha- 
bility, Biichi, coBUchi and parity conditions. 

4.2 Decidable problems for lasso shape words 

In this sub-section we consider the decision problems where, instead of restricting the probabilistic automata, 
we restrict the set of input words to lasso shape words. First, the processes induced by such words are simple: 

Proposition 8. Let Abe a PA, let w be a lasso shape word, and let a € A(Q). Then the process induced by w 
and a on Q is simple. 

Corollary 1. Let A4 be a finite state machine. Then for any w € T,^ generated by M, the process induced by w 
and a on Q is simple. 

The results of this section along with the results of the previous sub-section give us the following theorem. 

Theorem 3. Given a probabilistic automaton with parity acceptance condition, the question whether there is lasso 
shape word that is accepted with probability 1 (or positive probability) is PSPACE-complete. 

5 Structurally Simple Automata 

In this section we introduce the class of structurally simple automata, which is a structurally defined subclass 
of probabilistic automata on which every words induce a simple process. We show that the problems associated to 
this class of automata are decidable (the almost and positive problems are PSPACE-complete and limit problem is 
in EXPSPACE). We then show that this subclass of simple automata is closed under union and intersection, and 
finally show that structurally simple automata strictly generalizes HPA and #-acyclic automata. 

5.1 Simple automata and structural characterization 

Definition 3 (Simple Automata). A probabilistic automaton is simple if for all w € S"", the process {Pn}neN 
induced on its state space by w is simple. 

In ifm . given S ^ Q and a € S, the authors define the set 5 • a as the support of 5{S, a), and in the case where 
S ■ a = S, the set 5 • a* as the set of states which are recurrent for the homogeneous Markov chain induced on 
S by the transition matrix Ma. Next, they define the support graph Q^ of the automaton A as the graph whose 
nodes are the subsets of Q, and such that, given S,T C Q, the couple (S, T) is an edge in t/^ if there exists a € S 
such that S-a = Tor:S-a = S and S ■ a"^ = T. They present the class of #-acyclic automata as the class of 
probabilistic automata whose support graph is acyclic. 



Definition 4 ( lillj ). A probabilistic automaton A is ^-acyclic ifQ^ i^ acyclic. 

We now present a natural generalization of this approach. Given S Q Q and a finite word p G S*, let 5 • p = 
Supp((5(S', p)). If 5 • p = S, we define 5 • p* as the set of states which are recurrent for the homogeneous Markov 
chain induced on 5" by p (i.e. by the transition matrix {6{q, p){q')}q,q'£s)- 

Example 1. 

Consider the following probabilistic automaton A, We have Q-a = Q-a"^ = Q, and Q-b = Q-b"^ = Q. 

with state space Q = {s, t, u}. However, Q ■ {ab)"^ = {u}. 

a,. 5; 6,1 

a, .5 




Given a probabilistic automaton A, an execution tree is given by an initial distribution a G A((5), or a set of 
states A (1 Q, and a finite or infinite word p. We use the term execution tree informally for the set of execution 
runs on A which can be probabilistically generated when the system is initiated in one of the states of Supp(a) 
(or A), and when the word p is taken as input. 

Definition 5 (^-reductions). A ^-reduction is a tuple {A,B,p) where A, B C Q and p € S* are such that: 
(i) A^$, (ii) 3^0, (Hi) AilB = (/), (iv) (AU B) ■ p = Au B, and (v) [AUB)- p* = B. 

For simplicity, we may use the term ^-reduction for a couple (^4, p) where ACQ and p G S* are such that 

^ • p = ^ and A • p# / ^. 

Definition 6. An execution tree (a, p) is said to be chain recurrent for a probabilistic automaton A if it does 
not contain a ^-reduction. That is, for all pi, P2 G S* such that pi ■ p2 is a prefix of p, {6{a, pi), P2) is not a 
^-reduction. We write CRec(a) for the set of p a S* such that (a, p) is a chain recurrent execution tree for A. 

The following key lemma shows that for any probabilistic automaton A there exists a constant 7(^) > such 
that the probability to reach any state on a chain recun^ent execution tree is either or greater than 'y{A). Given 
a probabilistic automaton A, let e(^) be the smallest non zero probability which appears among the d{q, a){q'), 
where q,q' ^ Q and a G S. 

Lemma 1. Let A be a probabilistic automaton. For all q a Q, all p (z CRec(g) and all q' G Supp((5((7, p)) we 
have 5{q, p){q') > e^ where e = e{A). 

Definition 7 (Structurally simple automata). An automaton A is structurally simple if for all p G Tj* and C Q Q, 

if D Q C is minimal among the D C Q such that C — )• D, we have that D, p is chain recurrent. Here # — p 
intuitively denotes an iterated ^-reachability with the word p (details in Section \Q2\ of the appendix). 

We now prove that all the structurally simple automata are simple. We show that on a structurally simple 
automaton, given w £ T,^, the associated execution tree can be decomposed as a sequence of a bounded number 
of chain recurrent execution trees. The key Lemma [T] is then used to bound the probabilities which appear. 

Lemma 2. Let {pjf }neN be the process generated by a word w = ai, a2, ... G T,^ on a probabilistic automaton. 
Given n > 1 recall that w[l..n] = ai, ..., an. Suppose that there exists 7 > and N > such that for all n > N 
and all q G Supp((5(a, w[l..n])) we have 5{a,w[l..n])(q) > 7. Then the process is simple. 

We introduce the notion of sequence of recurrent execution trees in order to represent a process which may 
not be chain recurrent, but which can be decomposed as a sequence of a finite number of chain recurrent execu- 
tion trees. The length of the sequence measures the number of steps which do not belong to a chain recurrent 
subsequence, and will be useful to bound the probabilities which appear. Lemma|3]uses the key LemmaH] 
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Definition 8 (Sequence of recurrent execution trees). A sequence of recurrent execution trees is a finite sequence 

{ai,pi),p[,{a2,P2),P2, ■■■{ak,Pk) such that: 

• /9fc S S'^, and for i € [1; k — 1] we have Pi, p[ € S* 

• For all i € [2;k] we have Supp(aj) C Supp(5(ai_i,pi_i • Pi_i)) 

• All the execution trees (oi, pi) are chain recurrent 
The length of the sequence is defined as YliZ\ \p'i\- 

Given an execution tree (a, w), a subsequence of recurrent execution trees of (a, w) is a sequence of recurrent 
execution trees (ai, pi), p'^, (a2, /02), p'2, ■■■{ak,Pk) such that a = ai&ndw = pi ■ p\ • p2 • /O2 • • • • Pfc- 

Lemma 3. Let Abe a probabilistic automaton. Suppose that there exists i^ € N such that for all execution trees 
(a, p), there exists a subsequence of recurrent execution trees of length at most K. Then A is simple. 

Lemma 4. Suppose that A is structurally simple. Then for all execution trees {a, w), there exists a subsequence 
of recurrent execution trees of length at most 2'^'^'. 

The following follows from Lemma |3] and |4l 

Tiieorem 4. All structurally simple automata are simple. 

5.2 Decision problems for structurally simple automata 

For the following of this sub-section, ^ is a structurally simple automaton with state space Q and initial distri- 
bution a. We consider the complexity of the decision problems related to infinite words on structurlly simple PAs. 
The upper bound on the complexity in Theorem [5]follows from the results of Section |4l since the process induced 
on a simple PA by a word tf G S is always simple. The lower bound follows from the fact that the PA used for the 
lower bound of Section|4]is structurally simple. 

Theorem 5. The almost and positive problems are PSPACE-complete for parity conditions on structurally simple 
PAs. 

In Proposition 6 of ifTTTl . the authors show that if F C Q is reachable from a state qq in the support graph of A, 
then it is limit reachable from qq in A. A generalization of this result to the extended support graph gives half of 
Proposition |9] (details with complete proof in appendix). Theorem |6] follows from Proposition |9] and Lemma [T4l 
(details in Appendix IC.2l for the definition of the extended support graph). Theorem [T] shows that the limit parity 
problem is also decidable for simple automata. 

Proposition 9. Let Abe a structurally simple automaton, and let F Q Q. Then (1) F is reachable from Supp(a) 
in the extended support graph of A iff (2) it is limit reachable from a in A. 

Theorem 6. The limit problem is in EXPSPACEfor reachability conditions on structurally simple PAs. 

Theorem 7. The limit problem is in EXPSPACEfor parity conditions on structurally simple PAs. 

The following theorem establishes the decidability of the problem that given a probabilistic automaton whether 
the automaton is structurally simple. 

Theorem 8. We can decide in EXPSPACE whether a given probabilistic automaton is structurally simple or not. 

5.3 Closure properties for Structurally Simple Automata 

Given Ai = {Si, Tj,6i,ai) and A2 = {S2, S, ^2, a2) two structurally simple automata on the same alphabet S, 
the construction of the Cartesian product automaton Ai n A2 is standard. We detail this construction in appendix, 
along with the proof of the following proposition. 

Proposition 10. Let Ai and A2 be two structurally simple automata. Then A = Ai ^ A2 is structurally simple. 
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We prove that the classes of languages recognized by structurally simple automata under various semantics 
(positive parity, almost parity) are robust. This property relies on the fact that one can construct the intersection or 
the union of two parity (non-probabilistic) automata using only product constructions and change in the semantics 
(going from parity to Streett or Rabin, and back to parity; see ifTSl I3 for details of Rabin and Streett conditions 
and the translations). By Proposition [TOl such transformations keep the automata simple. 

Theorem 9. The class of languages recognized by structurally simple automata under positive (resp. almost) 
semantics and parity condition is closed under union and intersection. 

5.4 Subclasses of Simple Automata 

In this section we show that both 7^-acyclic automata (recall Definition ^ and hierarchical probabilistic au- 
tomata are strict subclasses of simple automata. 

Proposition 11. The class of structurally simple automata strictly subsumes the class of ^-acyclic automata. 

Another restriction of Probabilistic Automata which has been considered is the model of Hierarchical PAs, pre- 
sented first in |6|. Intuitively, a hierarchical PA is a probabilistic automaton on which a rank function must increase 
on every runs. This condition imposes that the induced processes are ultimately deterministic with probability one. 

Definition 9 (111). Given k £ 'N, a PA B = {Q, Qs, Q, 6) over an alphabet S is said to be a fc-level hierarchical 
PA (k-HPA) if there is a function rk : Q — > {0, 1, ..., k} such that the following holds: 

Given j € {0,1,..., k}, let Qj = {q £ Q \ rk(g) = j}. For every q £ Q and a G S, ifJQ = rk(g) then 
post{q,a) C LljQ<i<kQi and \post{q,q) D Q^J < 1. 

Proposition 12. The class of structurally simple automata strictly subsumes the class of Hierarchical PAs. 

It follows that our decidability results for structurally simple PAs both unifies and generalizes the decidability 
results previously known for ^-acyclic (for limit reachability) and hierarchical PA (for almost and positive Buchi). 

6 Conclusion 

In this work we have used a very general result from stochastic processes, namely the decomposition-separation 
theorem, to identify simple structure of tail cr-fields, and used them to define simple processes on probabilistic 
automata. We showed that under the restriction of simple processes the almost and positive decision problems are 
decidable for all parity conditions. We then characterized structurally a subclass of the class of simple automata on 
which every process is simple. We showed that this class is decidable, robust, and that it generalizes the previous 
known subclasses of probabilistic automata for which the decision problems were decidable. Our techniques 
also show that for lasso shape words the almost and positive decision problems are decidable for all probabilistic 
automata. We believe that our techniques will be useful in future research for other decidability results related 
to probabilistic automata and more general probabilistic models (such as partially observable Markov decision 
processes and partial-observation stochastic games). 
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Appendix 

A Details of Section |3] 

Details of Proposition |2l 

Proof, (of Proposition^.^e. present the proof of all three parts below. 

Assertion 1. Let i € {l,2,...,c}. We prove that r^ belongs to F^o- We first note that for all Nq ^ 'H and 

N > Nq, by definition of Tn, we have n„>Arr" € J^atq. Next, we note that {n„>ArT"}7vgN is an increasing 

sequence of sets of runs, and that the first point of Theorem [T] implies r^ = lim7v-i>oo (~^n>N t^- For all A^o ^ N' 

we have Tn^, is a cr-field, hence the limit of an increasing sequences of sets in T^o also belong to J^tvo ■ We get 

that for all Nq G N, we have r^ G T^q , hence the result. 

Assertion 2. We prove that r^ is atomic by using Proposition 2.1. of 191, which states the following result: 

• For any set T in Jqo, there exists a sequence L„ of subsets of Q such that, P"'-almost surely, we have 
limn^oo{r G ^2 s.t. r{n) £ Ln} = T. 
Here, "lim„_^oo{?' £ ^ ^.t. r(n) G L^} = F almost surely" means that the P^'-measure of the set of states on 
which the characteristic functions of the sets {r £ Q s.t. r{n) G L„} and F goes to zero as n goes to infinity. For 
sake of completeness, we prove this fact using the Martingale Convergence Theorem, as in ||3 (see for instance 
|[T3l for a presentation of the Martingale Convergence Theorem and the Levy's Law). 

Given n G N, let a{X'^,Xf, ..., X^) be the a-field generated by Xf , i G { 0, . . . , n }. Since F belongs 
to J'oo = n„6N-^n, the Levy's Law implies that, P"" almost surely, limn^oo^{T\a{X^ , Xf , . . .,X^)) = Ir, 
where Ip is the characteristic function of F. Since {X„, n > 0} is Markovian, we know that for all n we have 
¥{r\a{X^,Xf, ..., X;f )) = P(F[f7(X^)). Let < A < 1, and given n G N let L„ = {g G Q | P(F|X;f = q) > 
A}. Then, P^ almost surely, we have lim„_>oo{-'^n S L.^} = F, which proves the preliminary result. 

Now, let A G T^,. By hypothesis, P"'[A] > 0. Suppose by contradiction that < F'^IA] < P'"(r^). 
Let B = tI^\ A. We have A,B£ T^o, hence there exist Ln, L'^^, n G N two sequences of sets such that 
limn-^oo{^ € ^ I f{n) £ Ln} = A almost surely and lim„_j.oo{?' G ^ \ r{n) £ L'n} = B almost surely. Let N be 
large enough, and let q G Lj\f, q' G L'^ be such that : 

P[r G A I r{N) = q]>l 



4-IQI^ 



and 



[r £B\ r{N) =q']>l 



4-IQp- 

We prove that this contradicts the second point of Theorem [T] first, by the Pigeon Hole Principle, there exists a 
sequence qn, n > N of states in L^ such that 

lim„P[r(n) = g„| r{N) = q'] > ^— j^- 

Moreover, by the second point of Theorem [T] we know that 



hni^_).oo 



P[X- = qn\X^£jj;AX^ = q'] 



Thus, for n large enough, P[r(n) = qn\ r{N) = q] > — -— y. Hence, for n large enough, P[r A\ r{N) = q] > 

— r^-TTT. This is a contradiction. 

4-|Q|2 
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Assertion 3. The fact that, IP"^(IJ^=^ t^) = 1, is a consequence of the first point of Theorem[T} with probability 
one, after a finite number of steps, a run belongs to one of the J* and never leaves it. D 

Details of Proposition |3l We prove Proposition [3] 

Proof. Let J'^, ..., J'^ be a decomposition of Q'^ into jets, as in Theorem [T] Let A > be the threshold given by 
the definition of a simple process, for the process {/i^}„gN- For alH S { 1, 2, . . . , c } and n € N, let 

Jn = {q(^Jis.t.^i':l{q)>\} 

For all n G N, let i/° = J° U ULi(4 \ -^n). and let H'^ = j; for i G { 1, 2, . . . , c }. We claim that the 
decomposition of Q'^ into jets H = {H^, ..., H'^) satisfies the conditions of the proposition. 

The first point of the Proposition follows from the definition of the J^. The third point follows from the fact 
that the process is simple: the probability of the set of states whose measure is less than A goes to zero. We prove 
now the second point. 

Suppose that there exists no A^ G N such that the property is satisfied for the jet decomposition H. Then, there 
exists i G { 1, 2, . . . , c } such that for all N e'N, there exist n2 > m > N such that S{J^^ , w^^J % J\^. 

We write w = oq, ai, .... Since Q is finite, there exist i ^ j in { 1,2, . . . ,c} and q,q' G Q such that for 
an infinite number of n G N we have q G J^, q' G Jl, and 5{q,an){q') > 0. Since for n large enough we 
have fJ,n{q) > A for all q G J^, this implies that the probability of the set of runs which move from jet J* to jet 
J-' infinitely often is at least e • A, where e is the least non zero probability which appears among the transition 
probabilities given by the Ma, for a G S. This implies that the probability of the set of runs which stay inside one 
of the J*, i G { 1, 2, . . . , c } for ever after a finite number of steps cannot be equal to one. This contradicts the 
definition of the decomposition J^, ..., J'^. 

For the fourth point, the fact that the jets are mixing follows directly from the Theorem [H and the fact that a run 
does not leave J* once it has entered it after time N. D 

B Details of Section l 

Details of Proposition U We prove Proposition ID 

Proof. By |2| and |8|, the almost (resp. positive) safety and reachability problems are decidable for the general 
class of probabilistic automata, as well as the limit safety problem. The results of [2| and |8:] show that if one of 
the problems is satisfiable, it is satisfiable by a lasso shape word, and hence the simple version of the problem is 
satisfiable (by the results of our Section l4!2l ). As a consequence, we can use this result to get the decidability of 
the problems when we restrict to simple processes. The PSPACE-completeness follows from the results of |6|. 

The undecidability of the limit reachability problem comes from the results of [11] and [8], which show that it 
is undecidable for the general class of probabilistic automata, and from the following fact: Given a PA with state 
space Q, accepting states F C Q and e g]0; 1[, if there exists w £ T,^ such that P"'[{r [ r G Reach(F)}] > 1 — e, 
then there exists w' G T,'^ such that P"' [{r | r G Reach(F)}] > 1 — 2 • e and the process induced by w' is simple. 
For this we just have to consider any lasso shape word w = pi ■ P2 whose prefix word pi satisfies the 1 — 2 • e 
reachability condition. In Section 14.21 we see that the process induced by a lasso-shape word on an automaton is 
always simple, which concludes the proof. D 

Details of Proposition HI 

Proof. The proof is in three parts: first we present an equivalent formulation of the problem. Then we show that 
the equivalent formulation gives a problem which we can solve in PSPACE. Finally we give the PSPACE lower 
bound. 
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Equivalent formulation. In the following, p : Q — )• N is a parity function on Q, and $ = Parity (p). We prove 
that: (1) There exists w ^ Ti^ such that the induced process is simple and P^(<1') = 1 if and only if (2) There 
exists G C Q and pi,p2 G S* such that G = 5{a, pi), (5(G, p2) ^ G, and the runs on the Markov chain induced 
by (G, P2) satisfy <1> with probability one. We then show that the properties can be verified in PSPACE and also 
present a PSPACE lower bound. 

We show the equivalence (2)<^(1). The way (2)=^(1) is direct, since we will show in Section 14.21 that the 
process induced by a lasso shape word on any automaton is always simple. We prove that (1)=^(2). Let w = 
ai, ..., aj, ... be such that the induced process is simple and P^($) = 1. Using Proposition [3l let J^ , J^, ..., J"* be 
the decomposition of Q'^ into jets and let Nq ^'N, A > be such that: 

• Vn > A^o, Vi G { 1, 2, . . . , c }, Vg G 4: /i„(g) > A. 

• Vi G { 1, 2, . . . , c }, for all 712 > ni > iVo, we have 5(4^ , -w^^+i) Q J^^. 

• Each jet J*, i G { 1, 2, . . . , c } is mixing. 

Without loss on generality, since Q is finite, taking A'^o large enough, we can assume that the vector of sets of states 
(j]\r , ..., J^ ) appears infinitely often in the sequence {(4, ..., 4)}neN- As well, without loss on generality, we 
can assume that for all n > Nq and alH G { 1, 2, . . . , c }, all the states in J^ appear infinitely often among the sets 
J^, for m > Nq. Let i G { 1, 2, . . . , c }. Given q ^ Q, let 

<&y = {r G il I g G Inf(r) and p{q) = min p{q')} 

ij'Glnf(r) 

Clearly, for all q G Q, ^q £ Too- Since Q is finite, there exists qi £ Q such that P(r^ n <l>g.) > 0. By Proposition 
|2j T^ is atomic, hence r^ C <I)^.. Since the runs of the process satisfy the parity condition with probability one, 
p{qi) must be even. Moreover, for all n > Nq and all q G J^, we must have p{q) > p{qi). Indeed, such a q 
appears an infinite number of times in the sequence 4> by hypothesis, and always with probability at least A. 

Since r^ C $^., there exists rrij G N such that for all g G Jj °, there exists m < nii such that 6{q, w[Nq + 
l..'m]){qi) > 0. We define m = maxj^j 1^2,. ..,c} "^i' and m' >m such that 

Taking pi=w\})..Nq] and p2 = w[Nq + 1..Nq + m'] completes the proof. Indeed, when starting from the initial 
distribution, after reading pi, we arrive by construction in one of the sets J]^ , with i G { 0, . . . , c }. Starting 
from this state q, if the word p2 is taken as input, we go to set J^ ^^^, with probability one, visit qi with positive 
probability, and do not visit any state with probability smaller that p{qi). This implies that when starting from q 
and reading p2, we visit q^ with probability one, hence the result. 
Now, we argue the PSPACE upper and lower bounds. 

PSPACE upper bound. First, we show that we can verify the second property in NPSPACE, hence in PSPACE. 
The proof is in two steps. In a first step, we show that we can decide in NPSPACE whether, given G C Q, there 
exists pi G S* such that G = 8{a,pi). For this notice that, given G C Q, if there exists pi G S* such that 
G = 5{a, pi), then there exists p'^ G S* such that G = 5(a, p'^) and \p'^ j < 2''^'. Thus, we can restrict the search 
to words pi of length at most 2^^^ By guessing the letters ai , 02 , . . . of pi one by one, and by keeping in memory 
the set Ai = 5{a, ai, . . . , aj) at each step, we can check at each step whether A^ = G, and thus we can decide 
whether there exists such a pi in NPSPACE. 

In a second step, we show that, given G C Q, we can decide in NPSPACE whether there exists /92 G S* 
such that the runs on the periodic non-homogeneous Markov chain induced by (G, P2) satisfy <I> with probability 
one. For this, we refine the previous argument. Notice that this is equivalent to find p2 = ai, . . . ,0^ £ H* and 
A,B(ZQ such that: 

• p2 has length at most 2^' I'^l 
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• A,B partition G 

• Ais the set of recurrent states for the homogeneous Markov chain induced by p2 on G 

• B is the set of transient states for the homogeneous Markov chain induced by p2 on G 

• For all qo E A, for all the finite runs qQ,ai,qi,a2,q2, ■ ■ ■ ,ak generated with positive probability when 
initiated on q and when reading p2, the minimal value of the p{qi), zG{0, A: — l}is even. 

This can be checked in NPSPACE. Indeed, we can guess A, B, and the letters of p2 one by one, and at each step 
keep in memory the following sets: 

• The set of states visited at time i, i.e. Ei = 6{A U B,ai, . . . ,ai) 

• For all q £ A and all q' € 6{q, ai , . . . , Oj), the minimal p value of the paths visited between q and q'. Notice 
that this set has size at most \Q\. 

• For all q £ Au B and all q' G 5{A U B,ai, . . . , Oj), a boolean value Vi{q, q') which is equal to one if there 
exists a path between q and q' between the first step and step i, and which is null if not. 

At the end, we just have to check that Ej^ = G, that the minimal p-values of all the paths issued from A is even, 
that the set of states in A are recurrent for the chain, and that the states in B are transient. This can be done easily 
since we can recover the graph of the Markov chain on G from the values given by v\p^\. 

PSPACE lower bound. We prove now that the simple almost Buchi problem is PSPACE-hard. For this, we 
reduce the problem of checking the emptiness of a finite intersection of regular languages, which is known to be 
PSPACE complete by [14], to the simple almost Buchi problem, which is a particular case of the simple almost 
parity problem. The size of the input of Problem [T]is the sum of the number of states of the automata. 

Problem 1 (Finite Intersection of Regular Languages). 

Input: Ai, ...,Ai a family of regular deterministic automata (on finite words) on the same finite alphabet S. 

Question: Do we have C{Ai) r\ ... r\ C{Ai) = ^ ? 

Let Ai, ...,Ai be a family of regular automata on the same finite alphabet S, with respective state spaces Qi 
and transition functions 5i (where 6i{s, a){t) = 1 if there exist a transition from s to t with label a G S in Ai). We 
build a probabilistic automaton A = {Q, S', 6, a, F) such that the simple almost Buchi(F) problem is satisfied on 

AinC{Al)^^...r\C{Al) / 0. 

Let x be a new letter, not in S, and let S' = S U {x}. 

• Q is the union of the state spaces of the Ai, plus two extra states s and _L. That is Q = IJi=i Q'i U {^i -L}' 
where the Q[ are disjoint copies of the Qi. 

• The state ± is a sink: for all a G S', (5(_L, a){l.) = 1. 

• If u' is the copy of a non accepting state u of Ai, we allow in A the same transitions from u' as in Ai for u: 
if a G S, let 5{u' , a){v') = 1 iff v' is the copy of a state v £ Qi such that 5i{u, a){v) = 1. Moreover we add 
a transition from u with label x: 6{u, x)(±) = 1. 

• If u' is the copy of an accepting state u of Ai, i G [1; /], the transitions from u' in A are the same as in Ai, 
plus an extra transition 8{u' , x){s) = 1. 

• From state s in A, with uniform probability on i G [1; ^], when reading x, the system goes to one of the 
copies of an initial state of the ^j's. 

• For the transitions which have not been precised, for instance if a G S is read in state s, the system goes 
with probability one to the sink ±. 

• The initial distribution a is the Dirac distribution on s. 

• F = {s} 

Given p G C{Ai) PI ... PI C{Ai), the input word {x- p- xY satisfies clearly the simple almost Buchi(F) problem 
since a run visits s after each occurrence of x ■ p ■ x (the generated process is simple since we see in Section 
that any process generated on a probabilistic automaton by a lasso shape word is simple). 
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Conversely, suppose that there exists p € S"^ such that the induced process is simple and satisfies almost surely 
the Buchi(F) condition. 

• Since the only transition from s which does not goes to the sink has label x, the word p must start with letter 

X. 

• Since with probability one the runs induced by p visit infinitely often s, the letter x must appear infinitely 
often in p. Let p = x ■ p' ■ x where p' € S is non empty and does not contain the letter x. After reading 
x- p' -x, since the process cannot be in the sink _L with positive probability, it has to be on s with probability 
one. This implies that p' E C{Ai) D ... n £{Ai), hence £{Ai) n ... n C{Ai) / 0. 

This concludes the proof of the PSPACE completeness of our problem. We give an example of the last reduction. 

Example 2. Consider the following regular automata Ai and A2, and the associated probabilistic automaton A. 



a, b 




Figure 1. Automata Ai and A2 



a, b 




Figure 2. The probabilistic automaton A 

For instance, the word b- a- a belongs to C{Ai) £(^2)- We get that on A, the word {x -b- a- a- x)^ satisfies 
the simple almost Buchi{{s}) problem. 



This completes the details of the PSPACE upper and lower bound. 



D 



Details of Proposition |6l We prove that the simple positive parity problem is PSPACE-complete. 

As for the proof of Proposition [5j the proof is in three parts: first we present an equivalent formulation of the 
problem. Then we show that the equivalent formulation gives a problem which we can solve in PSPACE. Finally 
we give the PSPACE lower bound. We present only the first part of the proof in details, since the second and third 
parts are analogous to the proof of Proposition |5] 
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Proof. As before, let $ = Parity (p) where p : Q — )• N is a parity function. We follow a method analogous 
to the one for the simple almost parity problem: We prove that: (1) There exists u; G S'^ such that the induced 
process is simple and P^('&) > iff (2) There exists G C Q and pi, P2 G S* such that G C Supp((5(a, pi)), and 
5{G, P2) '^ G, and the runs on the Markov chain induced by (G, P2) satisfy $ with probability one. That is, we 
reach G with positive probability, and once we read p2 from a state in G we satisfy the condition almost surely. 

The way (2)=>(1) of the equivalence is direct. We prove that conversely, (1)=>(2). Suppose now that there exists 
such aw = ai, ..., Oj, ... . The induced processed is simple, so let J°, J^, ..., J"* be as given by Proposition [3l 
As before, without loss on generality, since Q is finite, we can also assume that the vector of sets (Jq °, ..., J^°) 
appears infinitely often in the sequence (J^, ..., J^), n € N. Moreover, we also assume that for all n > Nq, for 
alH G { 1, 2, . . . , m }, all the states in Jf appears in a infinite number of the sets J™, m > Nq. 

Let i€{l,2,...,c}. As before, an ultimate property is either satisfied or unsatisfied with probability one by 
the runs r G fi such that r(A'o) € J^. Thus, we can define qi £ Q as the state with minimal value for p among the 
states which are visited infinitely by runs in t°° with probability one. 

Since the runs of the process satisfy the parity condition with positive probability, there exists i€{l,2,...,c} 
such that p{qi) is even. Moreover, for all n > Nq and all q € Jf, as in the previous case, we must have p{q) > 
p{qi). Finally, there exists rn-j G N such that for all q £ J^ °, there exists m < rrii such that 6{q,w[NQ..m]){qi) > 
0. We define m' > mi such that 

(JjVg, •••, JivJ = [JNo+m'^ •••5 JNo+m') 

Taking pi = ti;[O..A'o] and p2 = w[No..Nq + m'] concludes the proof. 

The PSPACE upper and lower bound proofs are analogous to the proof of Proposition \5\ D 

Details of Proposition |7l We prove that the simple limit Biichi and coBiichi problems are undecidable. 

Proof. This is a direct consequence of the fact that the simple limit reachability problem is undecidable. The 
reduction from an instance of the simple Umit reachability problem is direct: we only delete all outgoing transitions 
from the accepting states in F, and transform them into self loops for all label a G S. We get a probabilistic 
automaton on which the simple limit Biichi and coBiichi problems are satisfied iff the simple limit reachability 
problem is satisfied. D 

Details of Proposition IH We prove Proposition [8] 

Proof. We just have to show that for any a E A((5) and p G S*, the process induced by p^ and a on Q is 
simple. Let {Xn}neN be the non-homogeneous Markov chain induced on Q by a and p^. Then for all i G 
{ 0, 1, . . . , IpI — 1 }, the chain {^n|p|+i}neN is homogeneous. The result follows from the classical decomposition 
Theorem of the state space of an homogeneous Markov chain into periodic components of recursive classes, and 
transient states. D 

Details of Theorem |3l We prove Theorem [3l 

Proof. By the results of Section|4l if the simple almost or positive parity problem is satisfied, then it is satisfied by 
a lasso shape word. Along with Proposition [H this implies that the simple almost (resp. positive) parity problem is 
equivalent to the question whether there is lasso shape word that is accepted with probability 1 (resp. with positive 
probability). Since the simple almost parity problem and the positive parity problem are both PSPACE-complete, 
the theorem follows. D 



17 



C Details of Section |5] 

C.l DetaUs of Sub-Section |5l] 

Details of Lemma [B We prove the key Lemma [T] 
Proof. Given [/ C Q and /j G S*, let 

5~\p){U) = {q^Q\5{q,p){U)>^} 

The following remark will be useful: given p = oi, ..., a„ € S*, given U '^ Q and i G { 0, 1, 2, . . . , n — 1 }, let 
Si = S~^{ai+i,...,an){U). Then we have: 

1. For alH G { 0, 1, 2, . . . , n - 1 }, 5{Q \ Si, a^+i, ..., a„) CQ\U 

2. For alH G {0,1,2,... ,n- 2},ii 6{Si,ai+i) C Si+i, then 6{a,ai, ...,ai){Si) = 6{a,ai, ...,ai+i){Si+i) 

3. Given i G { 0, 1, 2, . . . , n — 2 }, let fcj be the number of integers j G [i;n — 2] such that 6{Sj , Oj+i) ^ -Sj+i. 
Then, for alH G { 0, 1, 2, . . . , n - 2 }, 

S{a,ai,...,an)iU) > 5{a,ai, ...,ai){Si) ■ e^' 

The only non trivial point is the last one. It follows from the fact that for all p G S* and g, q' G Q, if 
5{q, p){q') > 0, then by definition of e we have 6{q, p){q') > el'^L 

By contradiction, suppose that there exists p G CRec(g) and U '^ Q such that U C Supp((5((7, p)) and 

%,/,)([/) <e2""^' 

We show that then we can write p = pi ■ P2 ■ P3 where pi, p2, Ps are such that 6{q, pi) can be partitioned into 
two subsets A and B such that {A, B , P2) i& & ^-reduction. This contradicts the definition of CRec(^). 
Let p = oi, ..., a/. Given i G { 0, 1, 2, . . . ,/ — 1 }, let: 

• Vl" = 5~^{ai+i,ai+2,---ai){U) n 5{q,ai, . . . ,ai) 

• W^ = {Q\Vl')r\5{q,ai,...,ai) 

Using the third point of the previous remark, since 6{q,p){U) < e^ , there exists a least k integers i in 
{ 1, 2, ...,/- 2 } such that (5(V;", Ci+i) ^ V^^, where k satisfies e'^ < e^"'"^' . Thus, A; > 22-IQI. Let ni, ..., n22.|Qi 
be the 22|<3I largest integers in { 1, 2, . . . , / } such that 6{Vj^, a^+i) ^ V^^^. 

By a simple cardinality argument, there exist i < j in { 1, 2, . . . , 2^I'3I } such that V^. = V^^ and W^. = W^.. 
Let pi = ai, . . . ,an,-i, P2 = 0,^, ■ ■ ■ ,an -1 and ps = a„ , . . . ,a„. Then we are in the following situation: 

q^l^W^^^l^W:^ 

P2 




yn P-2 yn 

That is, S{q,pi) can be partitionned into two subsets V^. and M^". such that 5{W^.,p2) ^ T^"-, d{Vn-,P2) Q 
V^^ U ly^^, and 5(K[^, /92) g F^'^. This implies that there exists A C V;"^ such that (k, {V;^^ \ A) U ^^^,^2) is a 
7^ -reduction. Since ^, {V^. \ A) U W^. is a partition of 6{q, pi), we get that the execution tree (g, p) contains a 
^-reduction. This is a contradiction since p G CRec(g). D 

C.2 Extended support graph 

In this sub-section, we introduce the necessary preliminary concepts before the formal definition of structurally 
simple probabilistic automata. These concepts can be listed as: 

18 



• A notion of Linked Graphs, and how to associate a Linked Graph to a word and an automaton. This is a 
technical way to represent transitions induced on a probabilistic automaton by an input word. 

• A notion of # — p reachability, which generalizes the notion of ^-reachability of IHH . 

• A notion of extended support graph of a probabilistic automaton, which generalizes the notion of support 
graph of [U |. 

We start with the notion of linked graphs. An example is given after the two following definitions to give more 
intuition. In the following of the sub-section, we call bipartite graph on Q a subset of Q y. Q. 

Definition 10 (Linked Graph). Given n > 1, a linked graph of length non Q is a sequence I = (Gi, . . . , G„) of 
n bipartite graphs: for alii G 1, . . . , n, the bipartite Gi is a set of couples of states of Q, i.e. Gj C Q x Q, such 
that for all i (z 1, ..,n — Iwe have Bi{X) = j4j+i(X), where: 

Vi G 1, .., n Ai{X) = {s € Q I 3t € Q s.t. (s, t) G d}; and Bi{X) = {t € Q | 3s € Q s.t. (s, t) G d} 

The set Ai = org(Z) is called the origin ofX, and the set Bn = dest(X) is called the destination ofX. 

Given a linked graph X = (Gi, . . . , G„), the compaction of X, written Comp(X), is the graph {V, E) where 
V Q Q and E C Q x Q, and such that for all s,t G Q we, have {s,t) G E if s G Ai,t £ Bn, and there exists a 
sequence of edges (si, ti) G Gi, (52,^2) G G2, . . . , (s„, t„) G Gn such that si = s and t„ = t. 

Given an edge-oriented graph Q = {V, E), a terminal component is a set of states A C V such that given any 
s,t G A, there exists a path between s and t in A. 

If dest(X) C org(X), we define Rec(X) as the set oft£V which belong to a terminal component of Comp(Z). 
Given s G org(X), we define Rec(s,Z) as the set of t G Rec(X) which are reachable from s in Comp(Z). 

Definition 11. Let n > 1, let A be a probabilistic automaton, and let p = oi, ..., a„ G S". Given A (^ Q, we 
define inductively the linked graph LG(/9, A, A) = (Gi, . . . , G„) induced by p and A on A as follows: 

• Gi = {(s, t) \ s G A and t G Supp{5{s, ai)))}. 

• For all i £ 1, . . .n — 1, Gj+i = {{s,t) \ s G dest(Gi, ..., Gj) and t G Supp{5{s, Oj+i))}. 
Example 3. Consider the following automaton: 

a:l,b:l 

n 



A 



6:1 



a:l 

o 



-5Q1I ^^ ^3 



b:l 

Let p = a ■ b ■ a. The state space of A is Q = {1, 2, 3}, and the linked graph X = LG(/3, {1, 2, 3}, ^) can be 
represented as: 

Ai Bi A2 B2 As B3 



1 — 

\ 


— >-l — 


— *-! 


1 — 


— *-i— 
\ 


— ^1 


\ 

2 — 


\-^2 


— ^2 — 


7^2— 


\ 
— ^2— 


v^ 


3 — 


— ^3 — 


— ^3 


3— 


— ^3— 


— ^3 
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Using a compact representation which avoids repetitions between Bi and ^Ij+i, we describe the previous linked 
graph as: 

1 ^1 1 ^1 




We have org(Z) = dest(X) = {1, 2, 3}, and Comp(X) is the graph: 



i- 

\ 




2 


>^ 


/ 


1 ■ ^ 


/ 

.s- 


-^3 



In this case, we have Rec(Z) = {2}, and for all s € {1, 2, 3} we have Rec(s,Z) = {2}. 

Definition 12 (Borders). A border of a linked graph I of length n is a couple of integers b = (ni,n2) where 
1 < ni < n2 < n and 3^2 (I) C A^^ (I). 

Definition 13 (Action of a border on a Unked graph). Let X = (Gi, . . . , G„) be a linked graph on Q, and let 
b = (ni, 122) be a border on I. We define LG(6,X) = {G'l, . . . , G'^), the linked graph induced by the action ofb 
on X, as follows: 

• For all i < ni — 1 we let G'^ = Gi 

• We define G'^^_^, G^^ , ..., GJ^ inductively. First, let J = G„i , ..., G„2- ^y hypothesis, we have dest(jr) C 
org(J'). 

- Given s,t £ Q, let {s,t) £ G^^_]^ if s € dest(G'^, ...,G^^_2) andt € Rec(s,J'). 

- Suppose that we have defined G'l, ...,G^ until i > ni — 1. Then we define G'-_^-^ = {{s,t) \ s G 
dest(G;,...,G9 and {s,t) G Gi+i}. 

Intuitively, the action of a border b = (rii, 712) on a Linked Graph corresponds to a ^-transition: we keep only 
the states of G^ which are recurrent for the sub-Linked Graph placed between ni and 77,2, and their successors. 

Example 4. Consider the same automaton as in example \3\ As before, let p = a ■ b ■ a and let I = 

LG{p,{l,2,3},A). Then b = (1,2) is a border on I since B2{T) = {1,2,3} = Ai[X). Then the graph 
LG(6,X) is the following graph: 



1 

\ 




1- 

/\ 


-^1 


\ 

2- 


V2 
\ / 


/ \ 
^2 


V' 


3- 


-^3 


3- 


-^3 
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Definition 14 (Chain of borders). Let X = (Gi, . . . , G„) be a linked graph on Q. We call chain of borders a 
sequence B = {{n\,n\), {n^^n^), . . . , (raf , 77-2)) of borders such that for all i € 1, . . .k — Iwe have that bi-^-l is 
a border o/LG(6j_i, LG(6i_2) • • • ) LG(6i,T) . . .)). We define LG(i?,X), the action of the sequence of borders B 
on X, as the linked graph V.G{B,X) = LG(6fc) LG(6fc_i, . . . , LG(6i,X) . . .)). 

Intuitively, the action of a chain of borders on a Linked Graph corresponds to a finite sequence of imbricated 
^-transitions. 

In the following definition we consider graphs whose nodes are subsets of Q, and whose edges are labeled by 
bipartite graphs I C Q x Q. Given C,D C Q and I C Q x Q, we write {C, I, D) for an edge labeled by the 
graph / between the vertice C and D. Given I C Q x Q,we define: 

left(/) = {s £Q\3teQ s.t. (s, t) £ 1} and right(/) = {teQ\3s £Q s.t. (s, t) G /} 

Definition 15 (Complete linked graphs). A complete linked graph on Q is a graph Q = {V, E) where V = 'P{Q) 
and the edges ofQ are labeled by bipartite graphs on Q such that for all edge {A, /, B) G E, we have A = left(/) 
and B = right(/). 

A path p = (Ai, /i, ^2, ..., /n-i, A„_|_i) on a complete linked graph naturally induces a linked graph 

(/l,...,/n). 

Definition 16 (^-reachability). Let C,D Q Q. Then: 

• Given p G S*, we say that D is ^ — p-reachable from C, written C ^ D, if there exists a chain of borders 



BonX= lG{C,p,A) such that D = dest{lG{B,X)) 
• We say that D is ^-reachable 
Let C,D Q Q and I 'Z Q- Then: 



We say that D is ^-reachable from C, written C —?■ D, if there exists p G S* such that C —?■ D. 



• Given p G S*, we say that D is ^ — p — I -reachable from C, written C ^ D, if there exists a chain 
of borders B on X = LG{C, p,A) and i G 1, ..., \p\ such that given V.G[B,X) = {Gi, ...,Gn), we have 
Comp(Gi, ..., Gi) = I and dest(Gi, ..., Gi) = D. 

#-i #-p-i 

• We say that D is ^ — I -reachable from C, written G —^ D, if there exists p £ T,* such that G —^ D. 

Example 5 (^-reachability). 

Consider the following probabilistic automaton: X = ({(1, 3), (1, 1)}, {(1, 3), (1, 1), (3, 3)}, 

-I'^^i {(1, 2), (3, 1), (3, 4)}{(2, 2), (1, 3), (1, 1), (4, 4)}, 
O {(1,1), (1,3), (3,3), (2,2), (4,4)}, 

^■- J ^^^ {(1,2), (2,2), (4,4), (3,1), (3,4)}) 

f) f) Now, ifh = (1,2), 62 = (5,6), andbs = (4,7). 

^ 3 ^ii^ *. 4 Then B = (foi, 62, ^3) i^ ^ chain of borders for X, and 

moreover we have: 




b:.5 

Let p = a-a-a-b-a-a-b. Let X = LG(1, p), let 



LG(6i,X) = ({(l,3)},{(3,3)}, 

, n 9^ A (r^a\ Ah (A 7\ Th {(3, 1), (3, 4)}{(1, 3), (1, 1), (4, 4)}, 

R ^ ' :C- ^^ ' A' : ^'f\'^'Z {(1,1), (1,3), (3, 3), (4, 4)}, 

B = (01,02,63) IS a chain of borders for X, and r,^ „^ ,. ^wo i\ (r, a\\\ 

moreover we have dest{lG{B,X)) = {4}. Thus {A} il-^^ ^J> (4>4j, ^^, ij, ^^,4j>j 

is ^-reachable from {1}. Notice however that {4} is Next 

not reachable from {1} in the support graph of A. \_G{(b^ bo) X) = (Ul 3)1 1(3 3)1 

Indeed we have {(3, 1), (3, 4)}{(1, 3), (1, 1), (4, 4)}, 
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{(1,3), (3, 3), (4, 4)}, 
{(4,4), (3,1), (3,4)}) 

And finally 

LG((6i,62,63),2:) = ({(l,3)},{(3,3)}, 
{(3,1),(3,4)}{(4,4)}, 



{(4,4)}, 
{(3,4)}) 



Thus, dest(LG(S,X)) = {4}, and {4} is #- 
reachable from {1}. Notice however that {4} is not 
reachable from {1} in the support graph of A. 



Definition 17 (Extented support graph). Let A be a probabilistic automaton. We define T-L^a = {V{Q),E), the 

4k— T 

extended support graph of A, such that given C,D Q Q and I <^ Q x Q, we have {C, I,D)^EifC — > D. 

Clearly T-La is a complete Unked graph. 

Proposition 13. We can construct Ha in EXPSPACE. 

We present the following Algorithm to compute a graph Gn in EXPSPACE, and the following Proposition 
shows that Gn = Ha- First, we need a preliminary Lemma, which can be proved using a simple counting 
argument. 

Lemma 5. Let Q be a complete linked graph. Let A (^ Q, let I Q Q, and let p = {Ai,Ii, ...,/„, ^n+i) be a 
path on Q such that Ai = A. Suppose that there exists a border h of (Ji, ...,In) and i € 1, ..., n such that given 
LG(6,Z) = (Gi,...,G„), we /jave Comp(Gi, ..., Gj) = /. Then there exists a path p = (^'^,IJ, ..., /^, 74^_,_^) 
on Q of length at most 2^^^ where A'^ = A, and a border b of {![, ...,I^) and i £ 1, ...,m such that given 
LG(6,Z) = {G'^, ...,G'^), we have ComviG'^, ...,G[) = L 

Algorithm 1 (Computation of the extended support graph). Let Abe a probabilistic automaton. 

• Let Gq = {V{Q),Eq) be the transitions labeled graph such that given A,BCQ and I (^ Q x Q we have 
(A, I, B) S Eq if there exists p ^Ti* such that 5{A, p) = B and Conip(LG(j4, p. A)) = L 

• Given i > 1, let Gi = {'P{Q),Ei) be the graph with transitions labeled by bipartite graphs such that 
given A,B (^ Q and I Q Q x Q we have {A, I, B) € Ei either if {A, I, B) € -Ej-i, or if there exists 
a path J = (Hi,H2, ...,Hn) in Gj_i (hence also a linked graph), and a border b of J, such that A = 
org(i7), and if V.Q{h,J) = (Gi,...,G„) there exists i € l,...,n such that B = dest(Gi, ..., Gj), and 
Conip(Gi, ■■■,Gi) = L Using Lemma\5l we can decide this fact in EXPSPACE. 

The graphs Gq,..., all have vertice in 1^(3), and their edges are labeled by bipartite graphs. Moreover, the 
sequence of the Gi is increasing (for the inclusion relation). The maximum number of different bipartite graphs 
between the subsets ofQ is bounden by |Q|I*5I • 21*^1 As a consequence, the sequence of the Gi must stabilize after 
some N < [Qjl'^l • 22|'3I, and we can build Gn in EXPSPACE. 

Proposition 14. Let Gq, ..., Gn be given by the Algorithm\J} Then we have Gn = Ha 

Proof. First, we prove by induction that for alH G 0, .., A^, all A, B C Q and I C Q x Q,if (A, I, B) is an edge 
of Gi, then {A, I, B) is also an edge of Ha- The case i = is trivial. Let i > l,\e.t A,B (1 Q and I Q Q x Q, 
and suppose that (^4, /, B) is an edge of Gi. If {A, I, B) is an edge of Gi-i we are done by induction hypothesis. 
If not, then there exists a path J = {Hi,H2, ...,Hn) in Gi-i and a border b oi J such that A = org(J'), and 
if \-G{b,J) = (Gi, ...,Gn) there exists i G 1, ...,n such that B = dest(Gi, ...,Gj), and Comp(Gi, ...,Gj) = /. 
Now, by induction hypothesis, J is also a path in Ha, and by definition of Ha, the linked graph LG(6, J) is also 
in Ha, which implies that {A, I, B) is an edge of Ha- 

Conversely, suppose that (G, /, D) is an edge of Ha- Let p G S*, i G 1, ..., \p\ and let B be the chain of borders 
onX= LG(D,,9,^)suchthatyli(LG(S,2:)) = D andif LG(B,X) = (Gi, ...,G|p|) wehave Comp(Gi, ...,Gi) = 
/. Then we can execute the border action on the set of graphs Gq, ••., G^ where k is the number of borders in B, 
to get the edge (G, /, D) in Gfe. This proves the result. D 
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C.3 Details of Sub-Section l5.1l - Continued 

Details of Lemma |2l We prove Lemma |2j 

Proof. For all n G N, we let A^ = Supp((5(a, w[l..n])) and Bn = Q\ An- By hypothesis, for all all n > A'^ and 
all q € An, we have f^nil) = <^(q^; w[l..n]){q) > 7. Moreover, for all n and all q £ BnWe have ii^{q) = 0. This 
shows that the process is simple. D 

Details of Lemma |3l We prove Lemma |3l 

Proof. Let p € S"^, and let {/UnjneN be the process induced on Q by p. By hypothesis, let 
(a, pi), p'l, (a2) ^2)5 P2) ■■■{<^k, Pk) be a sub-sequence of recurrent execution trees of (a, /)) of length at most K. 
That is, we have J2i=i \Pi\ ^ ^- ^y definition, for alH G { 1, ... A; - 1 } we have /)j G S* and p. G S*, and 
Pk G S'^. For all i G { 1, . . . , /c — 1 }, let a^ = 6{ai,pi). We are in the following situation: 



pi / P\ P2 I P2 Pk-1 Pk 

a ^ ai —T' a2 ^^ Oi2 ^ 0.3 . . . -^ o-k ^ 

We know that: 

• For alH G { 1, . . . , A: — 2 }, the execution tree {ai, pi) is chain recurrent 

• {ak,Pk) is chain recurrent 

We show that the process {/injnGN satisfies the hypothesis of Lemma[2] As before, let e = e(^) be the minimal 
non zero probability which appears among the values 6{q, a){q') when q,q' £ Q and a £ T,. Let A = e^ .By 
Lemma[T] for all q £ Q, all p' G CRec{q) and all q' G Supp((5(Q', p')), we have 6{q, p'){q') > A. We claim that 
for alH G { 1, . . . , A; — 1 } and all q G Supp(a^), we have a[{q) > (Min^ggupp(Q)a(g)) • A* • e^'*. We prove this 
result by induction on i: 

• The case i = 1 follows from the use of LemmaHJon the chain recurrent execution tree {ai, pi). 

• Suppose the proposition true until iG{l,...,A: — 2}. Letq' G Supp(a^_,_]^). then there exists q G Supp(aQ 
such that S{q,pi ■ p'i){q') > 0. Let q" G Q be such that 5{q,pi){q") > 0, and 6{q",p[){q') > 0. By 
the use of LemmaHJon the chain recurrent execution tree {ai,pi), we know that 6{q,pi){q") > X. By 
definition of e and K, we have that 5{q" , p[){q') > el'^il, hence 5{q" , p[){q') > e^ . We have a^^^{q') > 
ai{q) ■ 6{q, pi ■ p'j){q'). Since by induction hypothesis we have that ai{q) > (Mingggupp(„)a(Q')) • A* • e^"^, 
we get that a^_^i{q') > (MinggSupp(cj)a(5)) • A*+-^ ■ e^*+^, hence the result. 

Now, let N = X]j=i i\Pi\ + \Pi\)' ^i^^ l^t n > N. Since {ak,Pk) is chain recurrent, we can apply the same 
method for the chain recurrent execution ti^ee {ak, Pk)- As a conclusion, we see that the process {pn}nGN satisfies 
the hypothesis of Lemma |2] with the parameters N = ^^Zi {\pi\ + \Pi\) and 7 = (Minqggypp(^)a(g)) • A^ • e^'^ . 
This proves the result. D 

Details of Lemma |4l We prove Lemma |4l 

Proof We build iteratively the following sequences { Ai }jeN, { A'- }jeN, { Bi }ien, { Pi }ien, { p'i }ien, { ai }ien, 
{ Wi }jeN: 

• Let Ai = Supp(a), and Bi = 0. 

• pi is the longest prefix of w such that there exists A'l C Q such that Ai — I A'( and A" C Ai, and A[ is 

minimal among the set of A'( C Q such that Ai -^ A" and A" C Ai (minimal for the inclusion. If there 
exists several possibilities for A'l we just pick one of them). If the set of valid words is not bounded, then we 
let pi = w and the construction stops. If there exists no pi prefix of w such that there exists A'l C Q such 

that Ai — I A'l and A'l C Ai, then we let pi = e be the empty word and we continue the construction. 
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• Let wi be such that pi ■ wi = w. 

• If the execution tree {A[ ,wi) is chain recurrent, we stop the construction. If not, let p[ of maximal length 
be such that {A[, p[) is chain recurrent. 

• Let ai G S be the letter which follows pi • p[ in w. By construction, {A[, p[ • oi) is not chain recurrent, 
hence we can decompose p[ ■ oi as p[ ■ oi = p'l ■ p'/' and find Ui,Vi a partition of S{A[, p'{) such that 
{Ui,Vi,Pi) is a #-reduction and UiUVi = 6{A[,p'l) = S{A[,p[ ■ ai). We let A2 = Vi. Remark that 

A[ — !-> A2. Since Ai — I A[, this implies that Ai —4 A2. By definition of A'^, we have A2 7^ Ai. 

• LetB2 = 5{Ai,pi- p[-ai)\A2. 

• Let i > 1. Suppose that we have constructed the sets Ai, Bi,A[, . . . ^j+i, -Bj+i, and the sequence of finite 
words pi, wi,p'i,ai,pi, . . . pi,Wi, p'^,ai, p'{. We continue the construction as follows: 

- pi+i is the longest prefix of Wi such that there exists A'-^-^ C Q such that Aj+i — "^ ^i+i ^^id 

A^Yi ^ ^j+i» and A'-^-^ is minimal among the set of A"_^_-^ C Q such that vlj+i — ^ ^i+i ^^id 
A^'_i_]^ C Aj_|_i (minimal for the inclusion. If there exists several possibilities for A[_^_^ we just pick one 
of them). If the set of available words is not bounded, then we let pj+i = Wi and the construction 

stops. If there exists no pj+i prefix of Wi such that there exists A'/^^ C Q such that Aj+i — > A'-^^ 
and A^Yi — ^j+i' then we let pj+i = e be the empty word and we continue the construction. 

- Let Wi+i be such that pi ■ p[ ■ p'[ ■ ... ■ pj+i • tfj+i = w. 

- If the execution tree (^^^^,Wj+i) is chain recurrent, we stop the construction. If not, let p[^^ of 
maximal length be such that {A[^-^, P'i+i) is chain recurrent. 

- Let Cj+i G S be the letter which follows px-p'^-. . .-pi^^i-p'^j^^ in w. By construction, (^^^^ , /9^_|_]^ -aj+i) 
is not chain recurrent, hence we can decompose p[j^^ ■ Oj+i as p[_^_-^ ■ aj+i = p'-^-^ ■ pfj^^ ■ p'l'J_^ and find 
Ui+i, Vi+i a partition of 5{A'^J^^, p'l^^^) such that (?7j+i, Vj+i, /9.'|J is a #-reduction and Ui+iVJVi+i = 

<J(^-+i, p-'+i) = <J(^-+i, P-+1 • Oi+i). We let Ai+2 = Vi+i. Remark that A[^^ p^+^^+' ^.^^_ 3^^^^^ 

by induction we have Ai ^4 ^j+i and since by hypothesis Aj+i — ^ ^i+i » this implies that 

Ai — 5> ^i+2- By definition of ^^+1, we have ^i+2 / ^i+i, and by induction we 

have Ai^2 7^ Aj for all j < i + 1. 

- Let Bi+2 = 5{Ai,pi ■ p[- ai- . . . ■ pi+i ■ p\j^^ ■ Oj+i) \ Aj+i. 

Since there exists at most 2l'3l different subsets Ai of Q, the construction stops after at most 2l'3l steps. We get 
a sequence: 

f. Pl, P'l^i Pi p'i ai . Pi+\ 
A\ )■ >^ . . . > \-^ Aj+i > 

Where pi+i G S'^. Moreover, we now by construction that pj+i = wi, since by hypothesis the set of prefixes pj+i 

of Wi such that there exists ^^^.^ C Q such that ^j+i — ^^ ^i+i and A^_,_]^ C Ai^\ is not bounded. We can 
use the fact that A is structurally simple iteratively to show that this imply that there exists C C Ai^\ such that 
(C, Pi+i) is chain recurrent. Indeed, since A is structurally simple, to any finite length prefix pre/ of Wi such that 

-H- —'fyp p T 

there exists ^^^i C Q such that ^j+i — > -^'i+i and ^^+1 C Aj+i, we can associate Cp^-ej C ^j+i such that 

(Aj+i,pre/) is chain recurrent. Taking C C Ai^i which appears infinitely often among the Cpref concludes the 

point. 

For all i, Ai A- A[ is such that we can find Bi C Ai such that (i?i, pi) is chain recurrent. Since for all i we have 

p' 
that A'^ -4 AiJ^i is chain recurrent by construction, we get a sub-sequence of recurrent execution trees of (a, w) 

of length at most 2l'3l (only the sub-sequences which correspond to arrows -A may not contain a chain recurrent 

sub-sequence). D 
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C.4 Details of Sub-Section O 

Details of Proposition |9l We prove Proposition |9l 

Proof. We first show that (1)^(2). 

Let ^ be a PA, let p S S*, let C C Q, and let T-L^a be the extended support graph of A. Suppose C ^ D. Let 
B = (61, ...,bn) be a chain of borders and z G 1, ..., \p\ be such that given \-G{B, LG{C, p,A)) = (Gi, ..., Gn) we 
have D = dest(Gi, ...,Gi). Given j € 1, ...,n, let LG(5i, ...,6j, LG(C, p, ^)) = {G{,...,Gn). By induction we 
can show that for all j G 1, ..., n and alH G 1, ..., |p| we have that dest(G{, ..., G^) is limit reachable from G. 

We now show that (2)=^(1). Suppose that F is limit reachable from a in ^. First, if there exists p € S* such 
that Supp((5(a, p)) C F, then by definition F is reachable from Supp(a) in T-La- 

Suppose now that F is limit reachable from a in A, but that for all p G S* we have Supp(5(a, p)) ^ F- We 
define the following probabilistic automaton B with state space Q', alphabet S' and transition function as follows: 

• Q' = Q U {-L} where _L is a new state. 

• S' = S U {e} where e is a new symbol. 

• We keep the same transitions on ;B as in ^ when the labels are in S. 

• Given (7 G F, we add an extra transition with label e which leads to a state q £ Q with probability a{q). 

• Given q £ Q\ F,we add an extra transition with label e which leads to state ± with probability one. 

• From state _L, given any a G S' we loop with probability one on _L. 

We show that the automaton B is not simple. Since F is limit reachable from a, we can let {pn}neN be a sequence 

of finite words such that for all n G N we have 6{a, Pn){F) > 1 . We define w ^T!^ as: 

2" 

w = pi-e- p2 -e- p3... 

We claim that the process induced on the state space Q' oi Bhy w is not simple. First, notice that at any time, if 
the current distribution of the process is /3 G A((5') and the letter e is taken as input, then the probability to be in a 
state q £ Q at the next step is equal to a{q) * /3(F). As a consequence, the automaton A is not structurally simple. 
Given A; G N, let /3fc G A(Q') be the distribution on Q' that we get after having read pi ■ e ■ p2 ■ e . . . pk ■ e. By 
the choice of the pn, for all k we have fik{Q) > 1/2. By hypothesis, there exists q,q' £ Q such that q G Supp(a) 
and such that for an infinite number of n G N we have q' G Supp(J(g, pn)) and q' ^ F. Let 7 = a{q). We have 
found a couple q,q' £ Q such that: 

• For all k we have fik{q) > 7/2 

• Infinitely often, 6{q, pk){q') > and 6{a, pi ■ e . . . ■ e ■ pk){q') < -^ 

Such a couple q,q' invalidates the Proposition [3] which holds for simple process. Indeed, by Proposition [3l if 
infinitely often we have p^{q) > 7, then there exists A^ G N and 7' > such that for all n2 > ni > N, if 
/^ni (q) > 7 and 6{q, "w^^+i)!?') > 0, then /uj^^ il') > V- Thus, B is not simple. 

By definition, this implies that there exist G Q Q, D (^ D and p G S* such that G —> D and D is minimal, 
and {D, p) is not chain recurrent. Since the automaton A is supposed to be structurally simple, p must contain the 
letter e. Since {D, p) is not chain recurrent, D ^ {-L}, and in fact, since D is minimal, we must have that _L0 D. 
Let B = (61, ..., 6„) be the chain of borders such that dest(LG(B, LG(/0, G,A))) = D, letXo = LG(p, G,A), and 
foralH G l,...,nlet2i = LG(5i(LG(6i_i(...LG(6i(LG(G,p,^))). ..)))). Let A: = [/>[, and for alH G l,...,n we 
write Xi = (Go, ..., Gfc). Let p = oi, ..., a^, and let jo be the largest integer in 1, ..., A; such that Ojp = e. Then we 
have Aj^^ C F. As a consequence, F is reachable from Supp(a) in "H^. 

We get that in he new automaton, Supp(a) — > Supp(a), and more precisely that there exists a word p G S* 

which contains the letter e and which is such that Supp(a) -^ Supp(a). This implies that F is reachable from 
Supp(a) in "H^. 
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D 

Details of Theorem |7l We prove Theorem |7J 

Proof. We can prove the following, using the same kind of arguments as in the previous proofs: a structurally 
simple PA A satisfies the qualitative limit parity problem iff there exists a set of states ACQ such that: 

• A is limit reachable from Supp(a) 

• There exists p € S* of length at most 2l'5l such that A ■ p (^ A and the parity condition is satisfied on the 
Markov chain induced by A, p. 

This condition can be checked in EXPSPACE, using Theorem [6] D 

Details of Theorem |8l We prove Theorem [H 
We first need a preliminary Lemma: 

Lemma 6. Les A be a probabilistic automaton. Let C, D C Q, and suppose that C ^ D. Let E (^ C. Then 
there exists F (^ D such that E —^ F. 

Proof. Let p € S* and S be a chain of reduction such that given \-G{B, LG(C, p, A)) = (Gi, ..., G„) there exists 
i G 1, ..., 71 such that dest(Gi, ..., Gi) = D. Then by applying the same chain of border on LG(£', p, A) we get, if 
lG{B,lG{E,p,A)) = (G'i,...,G;,),thatdest(G;,...,G^) CD. D 

We now give the proof of Theorem [H 

Proof. Let ^ be a probabilistic automaton. We say that a subset C of Q is minimal if there exists no D C C such 
that C % D. Given C C Q minimal, let 

S{C) = {A C Q I 3p G S* s.t. A *-^^ C and Supp((5(A, p)) / C 

We claim that the following properties are equivalent: 

1. ^is simple 

2. Given C Q Q which is minimal, for all p G E*, possibly empty, we have Supp((5(C, p)) ^ S{C) 

This would prove the result since given C we can decide in EXPSPACE whether C is minimal, and we can 
compute S'(C) in EXPTIME using Algorithm [U 

We prove the claim. Suppose first (1), and let C C Q be minimal, and suppose that there exists p G S* such 

that D = Supp(5(C, p)) G 5(C). Then there would exist p' G S* such that D *^^' C and Supp((^(D, p)) / C. 
We get that 

C Ad *4^' C and Supp((5(i?, p)) / C 

Since C is minimal, by definition of a structurally simple automaton, we get that C,p ■ p' is chain recurrent. As a 
consequence, we should have D, p' chain recurrent, which is a contradiction. Hence (1) implies (2). 

Conversely, suppose (2), and let p G S*, C ^ Q, and Z) C C be minimal among the D Q Q such that 

C ^ D. Suppose that D, p is not chain recurrent. By Lemma[6j and since D is minimal, there exists p G S* 

such that D —^ D. Thus, if D, p is not chain recurrent we get D G S{D). This contradicts (2). Hence we get 
that (2) imphes (1). D 
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C.5 Details of Sub-Section U 

Details of the product construction and Proposition [lOl 

Given Ai = {Si, S, (5i, ai) and A2 = (5*2, S, 62, 0:2) two structurally simple automata on the same alphabet 
S, the construction of the product automaton Ai t>< A2 = {S, S, 6, a) is as follows: 

• S is the Cartesian product of 5*1 and S2- S = Si x 82- 

• Given (si, S2), (s'^, S2) G 5 and a G S, 5{{si, S2) , a){{s[, S2)) = 5i{si,a){s[) ■ (^2(^2, a)(4)- 

• Given (si, S2) G 5", a((si, S2)) = ai(si) • 02(82). 

Given s = (si, S2) G S, let pi(s) = si and P2(s) = S2 be the respective projections of s on the state spaces of 
Ai and ^2- 

We now prove Proposition [TOl 

Proof. For this we just have to remark that given p G S*, given j4 C 5, if we let Ai = {s £ Si \ 3t £ 
S2 s.t. (s, t) G 5} and ^2 = {* G S'2 | 3s G Sg s.t. {s, t) G S}, then we get ^ • p = (yli • p) x (A2 • p). Moreover, 
if A- p = A, then we have that A- p"^ = {Ai • p*) x {A2 • /?*)■ This implies that the structural simplicity condition 
is satisfied on Ai n A2 iff it is satisfied both on Ai and A2- 

a 

Details of Theorem |9l We prove Theorem |9l 

Proof. First, remark that the stability of the class of languages recognized by structurally simple parity automata 
under the positive semantics is trivial: we just consider a "union automaton" whose structure is the union of the 
structures of the two given automata, and whose initial distribution is a mix of the two given automata initial 
distributions. 

We consider now the stability of this class of language under the intersection operator. Let ^ be a structurally 
simple parity automaton. By defining a relevant set of accepting sets, we can transform its accepting condition 
to transform it to a positive Street PA which recognizes the same language. Since we do not change the structure 
of the automaton nor its transition function, the new automaton is still structurally simple. Now, given two Street 
PA with the positive semantics, using a classical product construction, we can construct a Street PA which, under 
the positive semantics, accept a language which is the intersection of the languages of the two Street automata. 
By Proposition [TOl this Street PA is still structurally simple. Finally, using a construction a la Safra, we can 
construct a parity PA which, under the positive semantics, recognizes the same language as the last Street PA. We 
can show that the construction a la Safra keeps the automaton structurally simple, since it can be seen as a product 
construction which does not add any probabilistic transition. We get the stability of the languages of positive parity 
PA under union and intersection. 

Remark next that PAs with positive parity semantics and PAs with almost parity semantics are dual of each 
others: given a PA A with positive parity semantics, by inverting the parity condition (taking a new parity function 
p' = p — 1), we get a new PA A' whose language is the complementary of C^^{A): C^^{A') = C-^'^{AY. 
As a consequence, if the class of languages recognized by positive parity PAs is stable under intersection and 
complementation, so is the class of languages recognized by almost parity PAs. D 

C.6 Details of Sub-Section lEi 

Details of Proposition [TTJ We prove Proposition [TT] 

Proof. By contraposition, let A be an automaton which is not structurally simple. Let C C Q, p g S*, and 
D (Z Q minimal such that C —^ D, and such that D, p is not chain recurrent. Let E = Ui>oSupp((^(D, p*)). We 
have E ^>- E. Let p = pi • p2 • ps be a decomposition such that (Supp((5(L', pi)), P2) is a ^-reduction. Then we 
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have that {Supp{S{E, pi), P2) is also a 7^ -reduction. As a consequence, we can build a cycle in the support graph 
of A, and thus A is not 7^-acyclic. 

Notice that every deterministic automaton is structurally simple. This implies that some structurally simple 
automata (for instance deterministic automata whose support graph contains a cycle) are not ^-acyclic. 

D 

Details of Proposition [I2l We prove Proposition [T2l 

Proof. Let ^ be a /c-hierarchical automata. Let C = Bi ^ B2 ^ ... ^ Bi = Bihe an elementary cycle in 
the support graph of A, i.e. a cycle which does not contain any sub-cycle. Let i,j € { 1, ...,/}, let g G Bi, 
and let q' G Bj. Then there exists p € S* such that S{q,p){q') > 0, and there exists p' € S* such that 
S{q,' , p'){q) > 0. This implies that rk{q) = rk(g'). This implies that there is not probabilistic transition in 
the cycle. As a consequence, C can not contain a ^-reduction. This proves that A is structurally simple. The 
automaton of Example [5] is structurally simple, but not hierarchical, which completes the proof. D 
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